Intro

This paper explains the methodology behind the National Cyber Power Index.

Theory

We identify seven objectives in cyber space:

  1. Surveillance = Surveilling and Monitoring Domestic Groups
  2. Defense = Strengthening and Enhancing National Cyber Defenses
  3. Information Control = Controlling and Manipulating the Information Environment
  4. Intelligence = Foreign Intelligence Collection for National Security
  5. Commercial = Commercial Gain of Enhancing Domestic Industry Growth
  6. Offense = Destroying or Disabling Adversary Infrastructure
  7. Norms = Defining International Cyber Norms and Standards

For each objective, we measure a states’ capabilities and intent to achieve said objective.

Each objective is weighted equally, therefore a country’s national cyber power score is the average sum score of each objective.

Formula

We produce two indexes, one only with cyber capabilities and one weighting the cyber capabilities with the intent

Cyber Capabilities Index \[NCPI = \frac{1}{7}\sum_{x=1}^7 Capability_x\]

Cyber Power Index

\[NCPI = \frac{1}{7}\sum_{x=1}^7 Capability_x*Intent_x\]

Data

We have 30 countries (observations) and for each country we have:
- 32 capability indicators (continuous and ordinal variables)
- 42 intent indicators (binary variables)

Open questions

In the following, we will focus only on the assessment and aggregation of the capability indicators to build the capability index.

1 Normalization technique
We have opted for z-standardizing the indicators before aggregating them.
- Are there any concerns with using this technique given our data? Another possible normalization technique would be the Min-Max approach.
- How comparable are the different indicators once they have been normalized? For example, can we compare the number of people in cyber military unites to to the number of content removal requests, or either of these to the number of top websites or top news websites registered by Google?
- Does our mechanism give us reasonable values down to one 10th of a unit (whatever the unit is)?

2. Choice of score range
We have set an arbitrary value for the mean of 50 and standard deviation of 20 (other scholars have taken a similar approach when building policy indexes)
- is there any argument against this approach?
- should be take a larger standard deviation so that the differences among countries are more evident?
- is the unit of analysis of the score one standard deviation? E.g., the U.S. gets a score of 70-some-odd, how can we best interpret this score and how does it relate to Germany that gets around 60 ? What units do we have?

3. Missing values
For North Korea information on 1/4 of the indicators is missing. These are all indicators related to internet and connectivity and we are trying to get reasonable estimates for them (based on other sources and expert assessments). Unfortunately we have only 30 countries (observations) and can not run more sophisticated techniques to input missing values.
We still would like to include North Korea in our raking, do you think this is reasonable?

4. Results from pairwise correlation
Discuss results from pairwise correlation of indicators.

5. Input Patrick Mair

Discuss Input from Patrick Mair and analysis with princals PCA.

Given that you only have 30 observations (and 30 indicators) you can’t apply any of the classical parametric psychometric techniques (IRT and friends) that you can do. If you want to do anything fancier than computing sum scores/means, you could compute use a method from the Gifi framework, as these techniques are exploratory (and non-parametric in some sense). To be precise, you can run a type of PCA called princals (see princals() function in the Gifi package) which is a PCA for mixed scale levels. You can find details in the Chapter 8 of my book “Modern Psychometrics with R” (Section 8.2.3 contains exactly much what you need), available through Hollis.
R code can be found here: https://github.com/pmair78/MPsychoR This method gives you (aggregate) PC scores which can then be used for subsequent modeling. If your indicators are unidimensional, you can use the scores from the first component; if it turns out that two dimensions are needed, then use the first 2 PC scores, etc.

Construction of Capability indicators

Step 1: datacollection

We have collected indicators of cyber power. Roughly a third of the indicators have been sourced in house

Step 1: subsetting the data

We have subsetted the data, so that for each objective we have a separate data set.
Each dataset contains indicators that measure said objective. Some indicators are a measure for different objectives and are therefore counted multiple times.

The latter is true for the following indicators

Step 2: normalize

We have normalized the indicators using z-scores. After performing normalization, the data have a common scale with a mean of 50 and a standard deviation of 20.

Since we have extreme values in our dataset, such as exceptional performance, z-score standardization is to prefer over min-max standardization

Z-Score Normalization Z-score normalization is calculated by subtracting the mean from an indicator value and then dividing by its standard deviation. If the standard deviation is calculated for a set of variables with a mean of 0 and then all values are divided by the standard deviation, the resulting set of values will have a standard deviation of 1 [27]. After performing normalization, the data have a common scale with a 0 mean and a standard deviation of 1. Since all Z-score distributions have the same mean and standard deviation, individual scores from different distributions can be directly compared. The advantage of this technique is that it provides no distortion from the mean, adjusting for different scales and variance. The output is dimensionless, and the relative differences are maintained due to the application of a linear transformation [42]. Z-score is preferred when extreme values exist in the dataset [20,32]. Although the technique does not fully adjust for outliers, the minimum and maximum values are not as influential as in other techniques such as distance to target. When extreme values are present in the original data, Z-score normalization takes these extreme values into account in a manner that does not distort their impacts on a composite indicator. In this way, an outlier, such as exceptional performance, is recognized and not ignored [13,27]. The Z-score technique is widely employed, including in the knowledge-based economy index [43] and the World Health Organization’s child growth standards index

Min-Max Normalization The min-max technique rescales data into different intervals based on minimum and maximum values. The advantage of this method is that boundaries can be set and all indicators have an identical range (0, 1). However, the normalized values do not maintain proportionality, and normalized values reflect the percentage of the range of maxas ( Xias)−min as(Xias) . This technique is based on extreme values (minimum and maximum), but because these two values can be outliers, the range of max and min strongly influences the final output. Another disadvantage is that the difference in variance is not fully eliminated [13]. Nevertheless, this technique is very popular and has been applied in the construction of many composite indicators, the best-known of which is the Human Development Index (HDI)

Step 3: aggregate

Given that we only have 30 observations (and 36 indicators) we can not apply classical parametric psychometric techniques (such as IRT ). To aggregate the indicators in a composite scale, for each country we have simply computed the mean of the sum of the indicators (per each objective).

There is an alternative (more fancy) aggregation method which we plan to test as well: Alternatively, we could use a method from the Gifi framework, as these techniques are exploratory (and nonparametric in some sense). To be precise, we could run a type of PCA called princals (see princals() function in the Gifi package) which is a PCA for mixed scale levels. The details are the Chapter 8 of Patrick Mair’s book “Modern Psychometrics with R” (Section 8.2.3 contains what we need), available through Hollis. R code can be found here: https://github.com/pmair78/MPsychoR This method gives (aggregate) PC scores which can then be used for subsequent modeling. If the indicators are unidimensional, you can use the scores from the first component; if it turns out that two dimensions are needed, then we use the first 2 PC scores, etc. (Input from Patrick Mair)

Pairwise correlation of indicators

1 Surveillance

Indicators:

  • laws (“”POS“”),
  • attack_surveillance (“”POS“”),
  • freedom_net (“”POS“”),
  • surveillance_firm (“”POS“”),
  • socials_use (“”POS“”),
  • internet_use (“”POS“”)

2 Defense

Indicators:

  • laws(“”POS“”),
  • shodan (“”NEG“”),
  • human_capital (“”POS“”),
  • cyber_firm (“”POS“”),
  • computer_infection (“”NEG“”),
  • mobile_infection (“”NEG“”),
  • internet_use (“”NEG“”) [REMOVE?],
  • broadband_speed (“”POS“”),
  • mobile_speed (“”POS“”),
  • ICT imports (“”NEG“”),
  • CERTS (“”POS“”)

3 Information Control

Indicators:

  • attack_manipulation (“”POS“”),
  • internet_use (“”POS“”),
  • socials_use (“”POS“”),
  • news_alexa (“”POS“”),
  • web_alexa (“”POS“”),
  • removal_google (“”POS“”)

4 Intelligence

Indicators:

  • attack_intelligence (“”POS“”),
  • tech_export (“”POS“”),
  • human_capital (“”POS“”),
  • cybermil_people (“”POS“”),
  • tech_firm (“”POS“”),
  • surveillance_firm (“”POS“”)

5 Commercial

Indicators:

  • attack_commercial (“”POS“”),
  • tech_firm (“”POS“”),
  • human_capital (“”POS“”),
  • cyber_firm (“”POS“”),
  • web_alexa (“”POS“”),
  • ecommerce_capita (“”POS“”),
  • tech_export (“”POS“”),
  • infocomm_imp (“”NEG“”),
  • patent_app_capita (“”POS“”)

6 Offense

Indicators:

  • attack_offense (“”POS“”),
  • tech_export (“”POS“”),
  • cybermil_people (“”POS“”),
  • military_strategy (“”POS“”),
  • cyber_command (“”POS“”)

7 Norms

Indicators:

  • laws (“”POS“”),
  • int_agreement (“”POS“”),
  • bilat_agreement (“POS”),
  • infocomm_imp (“”NEG“”),
  • tech_firm (“”POS“”),
  • tech_export (“”POS“”)

Sensitivity analysis

We compare the capability score to the GDP per capita.